Near-infrared spectroscopy (NIRS) is a useful technique for noninvasive measurement of oxygenation of the brain and muscle. However, no accurate, quantitative algorithms for continuous wave NIRS (CW-NIRS) have yet been presented dute to the following two problems. The first is that inhomogeneous tissue structure greatly affects measurement sensitivity. We previously reported on the influence of a fat layer on muscle oxygenation measurement and proposed a method for correcting the sensitivity. The second problem is that almost all algorithms for CW-NIRS have been experimentally determined, although an algorithm can be theoretically determined on the basis of diffusion theory if the mean optical pathlength in muscle in an in vivo state is known. In this study, we derived basic equations for a CW-NIRS algorithms based on diffusion theory, and we determined linear and nonlinear algorithms from mean optical pathlengths and validated them by results obtained from phantom experiments. For the determination of pathlength, the absorption and scattering coefficients of the muscle must be obtained by taking into account the influence of a fat layer. Laser pulses of 752 and 871 nm were applied to the forearms of subjects, and the temporal point spread function (TPSF) was obtained by using a streak camera. The absorption and scattering coefficients of the muscle were determined by fitting the measured TPSF with that obtained by a Monte Carlo model consisting of skin, fat and muscle layers. From these coefficients, the mean optical pathlengths at two wavelengths were obtained and the algorithms were determined.